Computing devices can be used to model user behavior for a given task and/or application. These models can represent data relationships and patterns, such as functions, algorithms, systems, and the like. The models may accept data input (sometimes referred to as an input vector), and produce output (sometimes referred to as an output vector) that corresponds to the input in some way. In some implementations, a model is used to generate a likelihood or set of likelihoods that the input corresponds to a particular value, e.g., a score. For example, a model may be used to determine what products should be recommended to a customer based on the likelihood that the customer will purchase each product. Different models are used for each individual task and/or application. These models use features specific to each task and/or application.
Numerous types of models may be used for these tasks and/or applications. Artificial neural networks (“NN”), including deep neural networks, are one type of model that can be used for modeling these tasks and/or applications. NN models are useful for solving problems that are difficult to solve using rule-based models, e.g., pattern-recognition, speech processing, face recognition, etc.
Scores in NN-based models are obtained by doing an NN forward pass. The forward pass involves multiplying large trained NN weight matrices, representing the parameters of the model, with vectors corresponding to feature vectors or hidden representations. The NN may progress from lower level structures to higher level structures. For example, for a NN trained to recognize faces in images, the input of the NN can comprise pixels. A lower level of the NN may recognize pixel edges, a higher level may identify parts of objects, such as eyes, noses, ears, etc., and an even higher level may recognize a face (or other object).
The parameters of a model can be set in a process referred to as training. A model can be trained using customer data that includes input data and the correct or preferred output of the model for the corresponding input data. The model can be used to process the input data, and the parameters of the model can be modified until the model produces (or “converges” on) the correct or preferred output.